Part0:前言

今天凌晨就開工直接弄到3:00，下班20:30繼續操作，過程有收穫和學習還不錯，測試結果似乎可行，只期望今晚跑所有資料能順利啦!(拜偷~~

Part1:今日目標

1.AlexNet目前進度和測試運算結果
2.競賽資料Debug紀錄

Part2:內容

1.AlexNet目前進度

經調整目前已經可以順利進行將預訓練模型使用在競賽資料，程式碼與測試結果如下:

(1)測試目的:確認模型運算過程沒有問題，才進行所有資料的大規模訓練

(2)測試版程式碼:運行4個Epoch，但每個Epoch只跑2個Batch的train_loader和2個Batch的val_loader，程式碼如下:

# 超參數設定
# Loss and optimizer
criterion = torch.nn.CrossEntropyLoss()
learning_rate = 0.005
num_epochs = 4
optimizer = torch.optim.SGD(model_alexnet.parameters(), lr=learning_rate, weight_decay = 0.005, momentum = 0.9)  

# Train the model
total_step = len(train_loader)
print(total_step)  # Total: 1570 batches

# Train & Validate model
total_step = len(train_loader)

for epoch in range(num_epochs):
    print("Epoch={}".format(epoch))
    print("=================="+"Train"+"==================")
    for i, (images, labels_tensor) in islice(enumerate(train_loader),1,3):
        print("train_batch_number: {}".format(i))
        # Move tensors to the configured device
        images = images.to(device)
        labels = labels_tensor.to(device)
        labels = labels.squeeze(1)  # Debug_1
        
        # Forward pass
        outputs = model_alexnet(images)
        labels = labels.type(torch.LongTensor)  # Debug_2
        loss = criterion(outputs, labels)  
        
        # Backward and optimize
        optimizer.zero_grad()
        loss.backward()
        optimizer.step()
        print("==== Finish an Train_batch ====")

    now = datetime.datetime.now()
    print(now)
    print ('Epoch [{}/{}], Step [{}/{}], Loss: {:.4f}'.format(epoch+1, num_epochs, i+1, total_step, loss.item()))
            
    # Validation
    print("=================="+"Validation"+"==================")
    with torch.no_grad():
        correct = 0
        total = 0
        for i, (images, labels_tensor) in islice(enumerate(val_loader),1,3):
            print("Validation_batch_number: {}".format(i))
            images = images.to(device)
            labels = labels_tensor
            labels = labels.squeeze(1)  # Debug_1, Debug_3 用來減少一個維度(一個中括號)
            outputs = model_alexnet(images)
            labels = labels.type(torch.LongTensor)  # Debug_2
            _, predicted = torch.max(outputs.data, 1)
            print("predicted={}, labels={}".format(predicted, labels))
            total += labels.size(0)
            correct += (predicted == labels).sum().item()  # Debug_3
            del images, labels, outputs
            print("==== Finish an Validation_batch ====")
     
        print('Accuracy of the network on the validation images: {} %'.format(100 * correct / total)) 
    print("==============================Finish an epoch==============================")

(3)測試運算結果如下圖:

• 橘色框和藍色框:第3和4個Epoch運算結果(Epoch編號從0起算)
• 粉紅框:Train的CrossEntropyLoss()計算數值
• 紅色框:模型在Validation資料上預測準確率(Accuracy:預測正確圖片張數/總預測圖片張數)
  

2.競賽資料Debug紀錄

關於上述程式碼共有3個Debug紀錄(Debug_1~Debug_3)和1個測試操作小技巧(Helper_1)，簡介如下:

(1)Debug_1

• RuntimeError: 0D or 1D target tensor expected, multi-target not supported
• Sol: labels = labels.squeeze(1) # squeeze(i)代表將tensor中對應的第i層維度進行降維
• Ref_CSDN

(2)Debug_2

• RuntimeError: expected scalar type Long but found Int
• Reason: LongTensor is synonymous with integer. PyTorch won't accept a FloatTensor as categorical target
• Sol: (change your target dtype) labels = labels.type(torch.LongTensor)
• Ref_Stackoverflow

(3)Debug_3

進一步確認資料:predicted & label維度不同，所以會變成矩陣乘積，造成正確率預算錯誤

• RuntimeError: The size of tensor a (10) must match the size of tensor b (4) at non-singleton dimension 0
• Reason: predicted和label這兩個tensor維度不一樣，所以運算predicted == labels會錯誤。
• Sol: 與Debug_1相同，使用labels = labels.squeeze(1)。

(4)Helper_1

• Method`m: Python: slices of enumerate
• 取一小部分的test_loader來Debug，若每次全都要重跑將很浪費時間
• Sol:

for i,elm in islice(enumerate(some_list),7,40):
    print i,elm

Part3:專案進度

完成Pytorch AlexNet Pretrained model於競賽資料上訓練和驗證設定。

Part4:下一步

確認全部貼標競賽資料運行時結果是否正確，若正確則整理實作細節，並蒐集進一步操作方式和進行下一個模型實作。

心得小語:
今天凌晨先開工是對的，不然會來不及啊啊啊~來不及寫心得了，充實的一天非常愉快!
(除了上述設定，在之前的程式碼class CustomImageDataset(Dataset)等也做些微調整，所以花蠻多時間呀
今日工時50min*6

To be both a speaker of words and a doer of deeds.
既當演說家，又做實幹家